Computing systems generally include peripheral devices. A peripheral device is a device that can extend, adapt, and/or modify the functionality of the computing system. Peripheral devices may be connected to a computing system internally or externally, and typically communicate with a computing system through one or more busses. The manner in which data is transferred on a bus is typically defined by a bus protocol. Examples of bus protocols include the Peripheral Component Interconnect (PCI) family of bus protocols (e.g., PCI, PCI-eXtended (PCI-X), PCI Express (PCIe), PCI derivatives such as Accelerated Graphics Port (AGP), etc.), among others.
In some computing systems, it may be desirable for one peripheral device to be able to use the resources of another peripheral device. For example, a PCI-based peripheral device may include a graphics processing unit (GPU). In many cases, a computing system may include multiple PCI devices, each hosting a GPU. In such systems, mathematical operations can be executed on multiple GPUs in parallel. When multiple GPUs are operating on the same data, one GPU may need access to the memory of another GPU to, for example, provide computational results or to obtain computational results.
Generally, when a peripheral device uses the resources of another peripheral device, the peripheral devices use a peer-to-peer communication system. “Peer-to-peer” generally describes a distributed architecture where tasks are partitioned between “peers” in a system. In a peer-to-peer system, the peers generally make at least a portion of their resources—such as processing power, disk storage, memory, and/or network bandwidth—directly available to other peers, without the need for central coordination by a host processor.
When the computing system uses a virtualized software environment peer-to-peer transactions may need to be sent to a host device in the system for address translation. In a virtualized software environment, the PCI endpoints may be configured and controlled by a guest operating system, executing a virtual machine. The virtual machine uses a virtual address space, which is typically different from the physical address space of the computing system. The guest operating system thus may have configured the PCI endpoints with virtual addresses, such that a peer-to-peer transaction from one PCI endpoint uses a virtual address to send a transaction to another PCI endpoint.
To provide address translations, the host device can include an Input/Output Memory Management Unit (IOMMU). A peer-to-peer transaction can be sent to the host device, where the host device translates the transaction's virtual address to a physical address. The host device can then send the transaction to the appropriate PCI endpoint.